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WATERMARK COMMUNICATION AND CONTROL SYSTEMS 

Related Application Data 
This patent application claims priority to U.S. Provisional Patent Application No. 
60/252,939, filed November 22, 2000, which is hereby incorporated by reference. 

Field of the Invention 
The present invention relates to use of watermarks to convey data to electronic 
systems, and is particularly illustrated in the context of enhanced television systems. 



Interactive television - a convergence between television and computers - has 
been heralded for a decade or more. To date, the technology has not found widespread 
acceptance. In part, this has been due to incompatibilities between television systems, 
such as NTSC and PAL. 

One key component of interactive TV systems is a data channel to accompany the 
video. Numerous techniques have been proposed - the most common of which is to 
encode data in the vertical blanking interval (VBI) of traditional analog TV signals. 
Another is to modulate data onto scanline 21 - a scanline that is usually positioned off- 
screen. Techniques that are commonly used with NTSC sometimes don't find favor with 
PAL, and vice versa. 

Much work has been done in recent years in the field of video watermarking - the 
science of conveying data through slight changes to the video information presented to 
the viewer ("in-band")- The changes are so slight as to be imperceptible to the viewer, 
yet can be recovered by suitable signal processing. Illustrative techniques are shown in 
the assignee's patent 6,122,403 and applications 09/138,061 and 09/164,859, and in 
patent documents WO99/45705, WO 00/04722. . . 

The focus of prior art video watermarking efforts has been to implement copy 
control functionality (e.g., to assure that copyrighted DVD video is not copied) and to 
provide some ownership marking of video content. 

In accordance with a preferred embodiment of the present invention, watermark 
technology is employed as a data channel in an interactive television system. If the 
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system relies on a consumer's set-top box (STB) to perform some of the system 
processing, the watermark processing operations can likewise be performed by the STB. 
Existing interactive TV systems can be modified to utilize a watermark communications 
channel by providing the requisite watermark processing function at a suitable layer in 
known interactive TV stack architectures. 

A similar approach, of providing watermark functionality as an additional 
component of known layered architectures, can likewise permit watermark-based 
communication channels to be employed in existing Ethernet networks. 

The foregoing and additional features and advantages of the present invention will 
be more readily apparent from the following detailed description. 



One emerging standard used in advanced television systems (and certain set top 
boxes) is known as ATVEF (Advanced Television Enhancement Forum - see 
www.atvef.com ; excerpts from this site are attached as Exhibits A and B). Through this 
standard, video content can produced once (using a variety of different tools), and can 
thereafter be distributed and displayed in a variety of environments (e.g., analog & 
digital, cable, satellite, distribution; display using STBs, digital TVs, analog TVs, PCs, 
PDAs, etc.). ATVEF is built on a number of other standards, including HTML 4.0, 
EcmaScript 1.1, and Multicast IP. In more technical jargon, ATVEF is a declarative 
content specification with scripting. 

Several familiar broadcast programs already employ this technology, including 
Wheel of Fortune, and Jeopardy, to enhance the viewer experience. The recently- 
introduced AOL-TV is based on ATVEF-compliant technology. 

At the consumer premises, a "presentation engine" is used to render the ATVEF 
content. One such presentation engine is known as ATSC's (Advanced Television 
Systems Committee) DASE, and sits on top of the application execution engine, with 
access provided via Java API calls. 

Many implementations of the ATVEF system employ Multicast IP for data 
transmission. In Multicast IP, data is conveyed in a part of the video signal that is not 
presented for display to the viewer. 

In order for video equipment to be compliant with the ATVEF standard, it must 
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recognize the data conveyed with the content signal, and render it in accordance with the 
ATVEF specification. A layered architecture is generally employed. 

Layered architectures are used in a variety of contexts. The lowest layer is 
commonly customized to the particular hardware being used. Higher layers are 
progressively more independent of the hardware - offering a hardware-independent 
interface for interacting with the system. By such approaches, software (and content) can 
more easily be used on a variety of different platforms, since the platform differences are 
masked by the layered architecture. 

ATVEF-compliant set top box architectures include a cross-platform 
communication stack having a layer that provides detection of the Multicast IP data. This 
layer analyzes the video data for the Multicast information, and relays the decoded 
information to higher layers that make use of such information in augmenting the 
consumer's experience. 

Likewise on the content origination side - a layered architecture is used. Such 
applications use stock IP protocols, such as Multicast or UDP. At (or near) the bottom of 
the stack different organizations have supplied a (Physical) layer to encode the signal into 
NTSC, PAL, DVB, etc. Traditionally, for each of these there is associated hardware 
(NABBTS encoder for NTSC, for example), that actually puts the data with the video. 

In accordance with one embodiment of the invention, watermark encoder/decoder 
functionality is provided at a similar layer in compliant systems. On the content 
origination side, a physical layer is provided to watermark video in any desired video 
format (typically in the spatial domain, but alternatively watermarking in the compressed, 
e.g., DCT or MPEG, domains), hence reducing the amount of hardware and software 
needed to operate with different formats. 

Likewise on the consumer side, a watermark detector is provided at a low level 
layer, serving to analyze the received video data for watermark information, and relay the 
decoded watermark information to higher layers that make use of such auxiliary 
information in augmenting the consumer's experience. (The video watermark decoder 
can be provided at the lowest - physical - layer, or at a higher level.) 

By arrangements like that detailed above, interactive TV employs watermark data 
- conveyed "in-band" in image content, to augment the consumer's experience. Rather 
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than implementing the technology differently for every origination system and set top 
box hardware (and associated STB operating system) on the market, the watermark 
functionality is desirably incorporated into a pre-existing layered communication 
architecture. By such approach, the installed based of content authoring tools, clients, 
5 and content is un-affected, and implementation is greatly simplified. 

To provide a comprehensive disclosure without unduly lengthening this 
specification, the patents and applications cited above are incorporated herein by 
references, together with application 09/571,422. 

Having described and illustrated the principles of the invention with reference to 
^ 10 illustrative embodiments, it should be recognized that the invention is not so limited. 

Sri 

0 For example, while the specification referred to a few examples of watermarking 

p I technology, the field is broad and growing. Any watermarking technology can be 

|* employed. 

hi 

Iff The implementation of the functionality described above (including watermark 

",15 decoding) is straightforward to artisans in the field, and thus not further belabored here. 

Conventionally, such technology is implemented by suitable software, stored in long term 

111 

memory (e.g., disk, ROM, etc.), and transferred to temporary memory (e.g., RAM) for 

'.y' : '* 

execution on an associated CPU. In other implementations, the functionality can be 
achieved by dedicated hardware, or by a combination of hardware and software. 
20 Reprogrammable logic, including FPGAs, can advantageously be employed in certain 
implementations. Set top boxes typically incorporate some or all of such elements. 

... v 

It should be recognized that the particular combinations of elements and features 
in the above-detailed embodiments are exemplary only; the interchanging and 
substitution of these teachings with other teachings in this and the incorporated-by- 
25 reference patents/applications are also contemplated. 

In view of the wide variety of embodiments to which the principles and features 
discussed above can be applied, it should be apparent that the detailed embodiments are 
illustrative only and should not be taken as limiting the scope of the invention. Rather, I 
: < claim as my invention all such modifications as may come within the scope and spirit of 

30 the following claims and equivalents thereof. 
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